Design and Implementation of A Web Mining Research Support System

by Jin Xu The evolution of the World Wide Web has brought us enormous and ever growing amounts of data and information. With the abundant data provided by the web, it has become an important resource for research. Design and implementation of a web mining research support system has become a challenge for people with interest in utilizing information from the web for their research. However, traditional data extraction and mining techniques can not be applied directly to the web due to its semi-structured or even unstructured nature. This proposal describes the design and planned implementation of a web mining research support system. This system is designed for identifying, extracting, filtering and analyzing data from web resources. This system is composed of several stages: Information Retrieval (IR), Information Extraction (IE), Generalization, and Analysis & Validation. The goal of this system is to provide a general solution which researchers can follow to utilize web resources in their research. Some methods such as Natural Language Processing (NLP) and Artificial Neural Networks (ANN) will be applied to design new algorithms. Furthermore, data mining technologies such as clustering and association rules will also be explored for designing and implementing the web mining research support system. IR will identify web sources by predefined categories with automatic classification; IE will use a hybrid extraction way to select portions from a web page and put data into databases; Generalization